Section: New Results

Uncertainty Modeling with Ontological Models and Probabilistic Logic Programming

Participants : Carlos F. Crispim-Junior, François Brémond.

keywords: probabilistic logic programming, activities of daily living, senior monitoring, ontological models,

We have been investigating novel probabilistic, knowledge-driven formalisms that can join the representation expressiveness of an ontology-based language with the probabilistic reasoning of probabilistic graphical models, like probabilistic graphical models and probabilistic programming languages. The goal is to support the representation of events (entities, sub-events and constraints) and hierarchical structures (event, sub-events) and at the same time be capable of handling uncertainty related to both entity/sub-event detection and soft constraints. Prior work in probabilistic logic provides support to reasoning either about uncertainty related to entity recognition (probability of entity x in the scene defined in ProbLog2) or to soft-constraint (relevance of violation of constraint $i$ to model $y$ as defined in Markov Logic). In our current work in partnership with KU university of Leuven, we have extended the ontological models of our vision pipeline (Fig.17) with probabilistic logic formalism proposed by ProbLog (Fig.18), a probabilistic logic programming language. Current results on the recognition of daily activities of seniors are promising as they improved the precision of our prior method by 1%. Further work will focus on extending our uncertainty models to be robust to constraint violations.

Figure 17. Pipeline for online activity recognition: given an acquisition camera (e.g. a Kinect), it firstly detects people using background subtraction algorithm, then it looks for appearance correspondence between people detected in the current frame with respect to past detections (past-present approach), and thirdly it recognizes the activities performed by each of the tracked people.

Figure 18. Temporal Inference using ProbLog engine. It takes as input deterministic observations and frame-wisely it recognizes the target events. Frame-events are aggregated into time intervals to create the time intervals of complex activities.